Data error correction system

ABSTRACT

A data storage system has a plurality of individual data storage units, each of which can undergo unpredictable independent failure. By dividing data blocks to be stored therein into a number of sub-blocks one or more less than the number of data storage units and creating a redundant data sub-block of the type permitting reconstruction of any one sub-block of data using the remaining sub-blocks and the redundent sub-blocks, and then storing each of the data sub-blocks and the redundant sub-block on a different one of the data storage units, it is possible to reconstruct any one failed data sub-block of a related group using the other sub-blocks. It is necessary to be able to detect the failure of the sub-block, and the preferred way is by a multibit error detection code appended to each sub-block, or by failure sensed within and by an individual data storage unit. The system is preferably implemented with disk drives functioning as the data storage units in view of the well-known tendency of such devices to occasionally generate an error which prevents reading a previously written record on one of them.

.[.BACKGROUND/INFORMATION DISCLOSURE.]. .Iadd.BACKGROUND OF THEINVENTION .Iaddend.

The device of choice today for non-volatile mass storage of data is themagnetic disk storage system. The type of magnetic disk storage systemof particular interest here is the so-called hard disk drive having, notsurprisingly, one or more rigid disks turning at a relatively highspeed. Each disk surface has suspended aerodynamically a few microinchestherefrom its own transducer device for reading and writing data on thedisk. In the larger data processing installations, there may be severaldrives all providing data storage for a single central computer. Forsome time, the reading or writing of several disk surfacessimultaneously has been contemplated in an effort to improve data ratesbetween individual disk storage units and the central computer. With therecent advent of large semiconductor memories, the difficult problem ofsynchronization of data transmission between the drives and the centralcomputer has been solved by the expedient of simply using suchsemiconductor memories as a buffer to compensate for differences inangular position of the disk.

While disk drive reliability has improved substantially over the lastfew years, the devices are nonetheless electromechanical and as suchliable to occasional failures. These failures may be caused by a circuitdefect which affects the readback function, in which case no data hasbeen lost. It is only necessary to repair the defective circuitry togain access to the data. If the failure comes at an inconvenient time,however, the delays may cause great expense for the users. If thefailure occurred in the writing circuitry or on the medium itself, thenthe data has been permanently lost. If the failure is a so-called headcrash where the heads strike and destroy the disk surfaces, then thatdata is permanently lost too. These cases usually are characterized bythe fact that only a single drive or drive controller is involved.

In many cases, the data stored on the disk drives in an installation ismuch more valuable than the drives themselves. This may arise in thesituation where the data represents a major investment in computer orhuman time. Sometimes the data has time-related value, say in areal-time environment or when printing time-sensitive materials such aspaychecks or management reports. Therefore, one must usually design suchstorage systems for high reliability since the cost of losing data dueto a drive failure is often unacceptably high. Accordingly there issubstantial motivation for avoiding such loss or delay of access to thedata.

The well-known prior art solution to some of these problems involves theuse of redundant data to detect and to correct data. The so-called rowand column error correction method uses row and column parity. That is,the bits of the data block are arranged in rows and columns (at leastconceptually) and a parity bit for each row and column is recorded withthe data block. A parity bit is chosen according to a preset rule toindicate for the bit group involved, such as a row or column, whetherthe number of binary 1's in the bit group is odd or even. Usually oddparity is used, where the parity bit is set to 1 if the number of "1"data bits in the group involved is even, so that the total number ofbits for a group is odd, thus assuring that at least one bit is presentin every case.

If parity in a single row and a single column is incorrect when a blockis read back from the recording medium one can assume with some degreeof assurance that the bit common to both the row and the column withincorrect parity is itself incorrect. The error can be corrected byinverting this common bit. It is usual to break the data into bit rowgroups of relatively short bytes of say 6 or 8 bits, with a row paritybit recorded for each byte. On the other hand, the column groups of bitsmay be quite long.

An alternative method for error detection and correction is representedby the family of so-called error correcting codes (ECC) which alsoinvolve the creation of a number of redundant bits for each data block.Common generic names for some of these are fire codes and Reed-Solomoncodes. These can detect many errors in a block of data, and allow inaddition several faulty bits in a block to be corrected. A well-knownlimitation of such ECC's is that they cannot correct more than a few biterrors in a block, nor can they correct more than one or two widelyspaced bit errors. Thus, they are particularly suited for correctingso-called .Iadd.random .Iaddend.burst errors where the errors areconcentrated within a few bits from each other as may occur on magneticmedia. Accordingly, it is the practice to use ECC redundancy within suchtypes of data storage unit as disk and tape drives .Iadd.for theprobable detection of massive errors and for the correction of randomburst error.Iaddend..

The readback electronics are also likely to produce occasional errors,but these are usually either random single bit errors widely spaced fromeach other, or errors spaced from each other at regular and relativelyshort intervals. These random errors are usually "soft", i.e. they donot repeat, and hence can be corrected by rereading the data from thestorage medium. Post readback byte parity redundancy (hereafter byteparity) may be used to detect these errors. By byte parity is meant theinsertion at regular intervals (i.e., with each byte), in the data justafter readback, a parity bit which provides parity error detection forthe associated byte. Regularly spaced errors are usually indicative of afailure after the serial to parallel conversion during readback. Sucherrors are not so easily corrected but can at least be detected by byteparity redundancy added to the data after it is read from the medium. Itis the usual practice to use EEC redundancy on the storage medium itselfand both byte parity and ECC redundancy during readback so as to providemaximum confidence in the integrity of the data manipulations duringreadback without a great amount of redundant data stored on therecording medium. Further, it is preferred to overlap the two sets ofredundant information so that no part of the data pathway is unprotectedby error detection/correction.

It is also known to use row and column error correction as describedabove in magnetic tape data storage systems. If the same bit in a numberof rows fail, this method allows reconstruction of the column soaffected. This usually is the result of a failure in the head orelectronics for the column since a tape medium defect is almost neverrestricted to a single bit position from row to row.

BRIEF DESCRIPTION OF THE INVENTION

The important insight in the invention to be described is that it ispossible to design a typical state-of-the-art data processinginstallation having multiple data storage units, so that failure of asingle storage unit occurs independently of and without affecting theavailability of similar units. For example, each may have its own powersupply and controller, now technically possible at modest additionalcost.

In this invention, a data block is split into a number of datasub-blocks, each of which is encoded for storage in a different datastorage unit (DSU) along with its own error detection and correctioninformation. A sub-block consists of a fixed number of bits organized ina sequence allowing each bit to be identified by its position in thesequence. For purposes of implementing this invention, each sub-blockbit is associated with the similarly positioned bits in the othersub-blocks to form a bit row. It is desirable (for purposes ofmaximizing speed of operation) that the storage units be approximatelysynchronized so that the sub-blocks all are read back withinapproximately the same interval and at approximately the same bit rate.

The system generates a redundant data sub-block for the data sub-blocksaccording to a preselected algorithm for which is data reconstructionalgorithm exists permitting reconstruction of any one data sub-blockusing the remaining data sub-blocks and the redundant data sub-block.Preferably, the redundant data sub-block comprises a set of parity bits,one parity bit being associated logically and positionally with each bitrow. Another, redundant, data storage unit stores this redundant datasub-block. During writing, it is convenient to generate the redundantdata sub-block bit by bit as the bit rows are supplied to the datastorage units so as to allow the redundant data block to be storedconcurrently with the data blocks. During readback of a particularblock, each redundant data block bit can be made available at about thesame time its row is.

The odds are extremely remote that two modern data storage units willfail simultaneously. Thus, when a single storage unit fails, the errordetection mechanism associated with it generates an error signal. Aspreviously stated, it is extremely unlikely that an error can occur inany column (or sub-block) without being detected by the associatedcolumn error detectors. Detection of an error in a sub-block is used toactivate data reconstruction means operating on individual rows. Eachrow having a parity error is corrected by inverting the bit in thecolumn for which the error signal was generated. The system of thisinvention in its preferred embodiment can correct several types ofmultiple errors.

By far the most likely multiple error failure mode is for a singlesub-block to contain all the errors. The physical basis for this is thata defect in the medium or circuitry of a single data storage unit maywell affect more than one bit in a sub-block, or for that matter, manysub-blocks being stored in the same storage unit. As mentioned earlier,since failure of even one of the data storage units is a rare event, thefailure of two within a short period of time is extraordinarily rare.

In the preferred embodiment, a byte error detection code is generatedfor indivdiual bytes encoded in each data sub-block signal provided by adata storage unit. This byte error detection code is generated accordingto a preselected byte error detection algorithm which includes as a partthereof steps by which certain data errors in each said byte may bedetected. Further, the data redundancy means in this embodimentgenerates according to the preselected algorithm, a sub-block of thetype allowing a byte to be corrected in a data sub-block by using theassociated bytes in the redundant data sub-block and the other datasub-blocks according to the preselected correction algorithm. Note thatthis approach allows correction of more than one error occurring indifferent sub-blocks of the same block so long as more than one of agroup of associated sub-block bytes does not have errors.

This apparatus is particularly suitable for implementation as a diskdrive data storage system. As mentioned earlier, it is advantageous toincrease data transfer rates by simultaneously reading and writingseveral data storage unit simultaneously. It is relatively easy todesign the system so that most disk drive failures are independent,i.e., are unlikely to cause any of the other drives to fail.

Accordingly, one purpose of this invention is to reduce the probabilityof losing data within a multiple storage unit data storage system to asmall fraction of the probability of an individual storage unit failing.

A second purpose is to allow storage units to be simultaneously writtenand read to increase data rates.

Another purpose is to avoid any interruption in operation of a dataprocessing system caused by failure of a single data storage unit (DSU).

Yet another purpose is to avoid the necessity for and expense ofemergency maintenance.

A related purpose is to allow maintenance necessitated by failure of anindividual data storage unit to be deferred to a scheduled maintenancetime, typically much less expensive.

Another related purpose is to allow a failed DSU to be taken off-lineand repaired while the rest of the system functions with the errorcorrection active and so permit uninterrupted system operation duringsuch repair.

Other purposes will become evient from the descriptions which follow.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a simplified system incorporating theteachings of this invention.

FIG. 2 is a detailed block diagram of the data reconstruction circuitry.

DESCRIPTION OF A PREFERRED EMBODIMENT

1. General

The preferred system disclosed below has fewer than the number of datastorage units one would usually select. However, the number selected (4)accurately illustrates a system operating according to the teachings ofthis invention and avoids the confusion which adding the likely morepreferable 8 or 9 data storage units might create. Note that manydifferent configurations of this invention are possible. The variousdetails of this embodiment are merely illustrative, and are not intendedto exclude others. For example, many variations in the logic circuitryare possible to implement the functions described. As the explanationproceeds, possible variations will be mentioned on occasion, however, soas to allow the reader to understand the many specific configurationswhich the invention may have.

This invention is described with reference first to FIG. 1 which is ablock diagram comprised of individual data storage subsystem blocks. Itis believed that the function(s) of individual blocks are described withdetail more than sufficient to allow someone with skill in the art toeasily understand and construct the invention. Many of the individualblocks represent one or more microcircuit elements commonly availabletoday. Other elements, such as data storage unit (DSUs) 19a, b, c, d arewell-known devices which may be, for example, disk drive units asmentioned previously. Individual blocks are connected by data paths overwhich individual bits represented by electrical pulses flow. Unlessindicated otherwise by a small circle with a number in it (e.g., ref.No. 27) specifying the number of parallel lines represented, it shouldbe assumed that data flow on an individual path is serial, i.e.,individual bits are provided sequentially to the destination block orthat the path carries a control signal of some type.

It should also be understood that no attempt is made in these FIGS. toshow the explicit timing relationships needed to allow the system tofunction properly. Such timing considerations are well understood in theart and hence need not be discussed in detail. Because of this, it isfelt that including detailed timing is as likely to obscure as toclarify the elements and theory of the invention. Furthermore, thewritten description either explicitly or implicitly establishes all ofthe timing relationships necessary to understood and implement theinvention.

It will be noticed that FIG. 2 contains much more detail than doesFIG. 1. This is because FIG. 1 is concerned mostly with the writing ofthe data in a format permitting its correction by the apparatus shown inFIG. 2. The correction or reconstruction of the data is an inherentlymore complex problem than mere recording of the original data with theredundancy needed to permit the correction. Thus to adequately disclosethe invention it is necessary to describe the readback apparatus ingreater detail than the writing apparatus.

2. Writing

Turning first to FIG. 1, data blocks, each comprising a fixed number ofbits, can be considered to become available one at a time from anexternal data source on a data path 11 when the system of FIG. 1 is idleor otherwise able to accept a block. It is convenient to assume thateach block has the same number of bits in its, typically in thethousands or tens of thousands of bits. The data on path 11 is receivedby a block divider 10 which divides the data block into three sub-blocksof equal length which are transmitted on data paths 12a, b, c to ECCgenerators 13a, b, c respectively. Block divider 10 can be designed tooperate in one of two modes, either of which are acceptable. In thefirst mode, a serial order is established for all the bits in the datablock on path 11 and then the first, fourth, seventh, etc. are placed onpath 12a; the second, fifth, eighth, etc. on path 12b; and the third,sixth, ninth, etc. on path 12c. Alternatively, block divider 10 candivide each data block into sequential groups of bits, or bytes, placingeach first group sequentially on path 12a, each second group on path12b, and each third group on path 12c.

Further, it is convenient to specify a sequence for the bits comprisingeach sub-block, and to associate the bits occupying the same position inthe sequence in each sub-block. Each such group of bits, each bit in agroup being from a different sub-block, will be referred to as a rowhereafter, from the analogy to a bit matrix where each sub-blockcomprises a column. In this embodiment, the bits comprising each row areissued simultaneously by block divider 10. It is immaterial whether bitsare provided serially or in parallel on paths 12a, b, c, although theelements receiving signals on these paths must be compatible with theformat chosen.

ECC generators 13a, b, c are substantially identical devices whichgenerate error correction and detection data for each data sub-blockwhich is received on their respective input data paths 12a, b, c. TheECC code for each sub-block is generated as the sub-block is received,and the data is passed through the ECC generator involved and encoded ina signal placed on an associated path 14a, b, c. At the end of the datasub-block, the ECC code value has been determined and is encoded andappended to the signal for each data path 14a, b, c. As mentionedearlier, the algorithm used by ECC generators 13a, b, c provides a veryhigh likelihood of detecting any errors in a data sub-block.

Row parity generator 15 also receives the data sub-blocks row by row onpaths 12a, b, c from block divider 10. Recall that the data bits formingeach row are simultaneously presented in the signals on paths 12a, b, c.Parity generator 15 determines the parity of each row of bitssimultaneously presented to it on paths 14a, b, c and a few tens ofnanoseconds later provides a signal encoding this parity on path 12d,thereby preserving approximate synchronization between the data on paths12a, b, c and the associated row parity bits on path 12d. As a practicalmatter a few tens of nanoseconds are negligible compared to the durationof one bit interval on paths 12a, b, c. ECC generators 13a, b, c, d canall be considered to be similar devices having identical internalspeeds. Thus, data storage units (DSUs) 19a, b, c, d in effectsimultaneously receive each row and the row parity which has beencalculated for it by parity generator 15. If parity generator 15 is soslow that it destroys the synchronism between the bit rows and theirindividual row parity bits, then it is a simple matter to deal with thisproblem by, for example, inserting signal delays in paths 14a, b, c.

While each row with its parity need not, in the general case, bepresented simultaneously to the DSUs 19a, b, c, d, it is usuallypreferable to do so, so that each DSU 19a, b, c, d, is active at thesame time, increasing the bit storage rate. In systems which use thepreferred disks as the media in the storage units, synchronizing thedisk rotation results in very large increases in both storage andretrieval speed if the bits of each row are simultaneously presented totheir storage units.

At the time the data block to be stored in DSUs 19a, b, c is placed onpath 11, a signal is also placed on the read/write control path 25 whichspecifies that writing or storage of data is desired, and also specifiesthe physical location on the disks at which the data block is to bestored. The source of this signal may be a CPU (central processing unit,i.e. computer) which uses the system of FIG. 1 as a peripheral device,or it may be a system controller or may have parts supplied by both.

The purpose of the invention is to deal with a failure of one of DSUs19a, b, c by using the redundancy supplied to the system by DSU 19d torecreate the data. To justify the cost of an additional DSU, the unitsmust be relatively cheap in comparison to the data to be stored.Further, failure of one unit must in most cases be independent offailure of others. That is, the cause of a failure must usually be ofthe type which causes only a single one of the units to fail, so as toallow the system of this invention to recover or recreate the data.Examples of such kinds of failures are power supply and fuse failures,logic and signal processing failures, head and medium failures in themagnetic tape and disk systems, bad cabling connections, etc.

Examples of non-independent failures which the system of this inventioncannot correct are power failures which cause all units to failsimultaneously, or failure of controller hardware common to all theunits. But if the failure is one where an individual one of the unitsfails and the other units continue to perform normally, then thisinvention can make a useful contribution to overall system reliability.

Therefore, I prefer that each DSU have its own controller so thatcontroller failure is localized in a single storage unit. Such DSUs failrelatively rarely, and failures are for the most part independent ofeach other.

If DSUs 19a, b, c, d are magnetic or optical disk drives, as ispreferred, synchronizing the disk rotation to each DSU allows bit spacesequences on one disk medium to be permanently associated with similarsequences on the other DSUs' media, so that associated sequences passbeneath their read/write heads during nearly the same time interval.Such synchronization has the further advantages of allowing simplifiedreadback and true parallel data operation.

The remainder of the description will proceed with the assumption thatthe preferred disk drive units are employed as DSUs 19a, b, c, d. DSUs19a, b, c, d all receive and store each set of three rows bits and theirassociated parity bit very nearly simultaneously. As successive sets ofrows and the associated parity bits are presented to DSUs 19a, b, c, d,these too are stored so that at the end of the sub-blocks, the bits arearranged on the disks within the DSUs 19a, b, c, d in serial fashion.The individual sub-blocks are followed by the aforementioned ECCinformation which is also stored serially on the DSU's disks. Thus, whenwriting of a block has been completed, each sub-block has been seriallystored with its ECC information data appended. Further, because of thesynchronization of the individual DSUs' spindles, when the read/writeheads are positioned in the tracks storing the sub-blocks involved, thebits of each individual row will appear beneath the respectiveread/write heads at very close to the same instant.

It is usually the case that a particular data block is to be stored at apredetermined physical location on the disks of DSUs 19a, b, c, d. Thus,the data block must be presented to block divider 10 at a timesynchronized with the angular position of the spindles which carry thedisk media within DSUs 19a, b, c, d. Typically, the data source isitself signalled to begin transmitting the data block to be stored whenthe read/write heads have been properly positioned in the desired datatracks and the disks' angular positions are such that the writingsignals appear on the read/write heads as the desired physical lengthsof the tracks are passing beneath the heads. Such synchronization andcoordination between the transmission of data from the source and thedisk(s) on which it is to be stored is well known.

3. Reading

During reading, control signals encoding the location of the desireddata block issued to the individual DSUs 19a, b, c, d on path 25 causethe read/write heads to be positioned on the tracks containing thesub-blocks of the desired data block. Further, the read/write signal onpath 25 specifies the desired function as reading. As the individual bitspaces move past the read/write heads, each of the DSUs 19a, b, c, dencode in a raw data signal carried on paths 16a, b, c, d respectively,the bits of the sub-block stored in the track spaces specified by theread/write signal. Bits in the raw data signals are accompanied by clock(CLK) signals on paths 15a, b, c, d, as provided by the DSU 19a, b, c, dinvolved. A set of serial to parallel circuits 26a, b, c, d receives theraw data and clock signals from their respective DSUs 19a, b, c, d andassembles each successive set of 8 bits into 8 bit parallel byte signalson paths 17a, b, c, d followed a very short fixed internal later by abyte clock signal on the associated path 22a, b, c, d.

Byte parity generators 18a, b, c, d receive the 8 bit bytes on paths17a, b, c, d respectively and generate an odd byte parity bit for thebyte received, encoding this parity bit in the signals on paths 24a, b,c, d respectively. Byte parity generators 18a, b, c, d are of the typewith such great internal speed relative to the time that a particular 8bit byte signal is available on paths 17a, b, c, d that each 8 bit bytesignal and its associated byte parity bit can be treated as a single 9bit byte. This is symbolized by the combining of the parity signals onpaths 24a, b, c, d with their related byte signals on paths 17a, b, c, dto form 9 bit byte signals as encoded on paths 21a, b, c, d. This, theclock signal on the respective clock signal path 22a, b, c, d identifiesthe times at which individual data and row parity bits are present onpaths 21a, b, c, d respectively. If errors occur in the data duringlater processing, testing this row parity is very likely to reveal sucherrors, and the capability of the error correction system to bedescribed allow errors in different sub-blocks to be corrected in manycases.

A data recovery system 30 receives these data and row parity signals andprovides an output signal on path 62 encoding the data block originallysupplied on path 11, correcting those errors which are correctable.Internal faults sensed by DSUs 19a, b, c, d are indicated to datarecovery system 30 on their respective fault signal paths 23a, b, c, d.In many cases, this system can also recover from complete loss of dataon one DSU 19a, b, c, d, as indicated by a fault signal on a path 23a,b, c, d.

4. Error Recovery

FIG. 2 discloses the details of system 30 which allows thereconstruction of an entire data block stored on DSUs 19a, b, c in spiteof the presence of one or more otherwise uncorrectable errors in, oreven the unavailability of, a constituent sub-block stored on any one ofthe DSUs 19a, b, c. The earlier-mentioned read command on path 25 alsosignals a control logic element 50 to begin a read sequence, the stepsof which will be described in conjunction with the description of thevarious elements shown in FIG. 2.

The major elements at the input side of the readback circuitry aresub-block buffers 52a, b, c, d, which store each entire sub-block asthey are received on paths 21a, b, c, d from DSUs 19a, b, c, drespectively. Sub-block buffers 52a, b, c, d are similar devices fromwhich the data sub-blocks are read and corrected if necessary. The byteparity, DSU fault signals, and the appended ECC information may all beused to determine need for corrections. Their use will be explainedusing buffer 52a as an example. Buffer 52a has an internal pointerregister for addressing its bit locations. This internal register isinitially cleared by a RESET ADR (ADdRess) signal on path 66 generatedin response to a read command on path 25. The internal pointer registeris incremented by one by each clock (CLK) signal pulse on path 68a. Whenthe read/write select (R/W SEL) signal on path 65 is set to a logical 1,it places buffer 52 a in write mode and individual 9 bit bytes can beloaded into buffer 52a via data path 21a and stored or written in thelocation in buffer 52a specified by its pointer register. Successiveclock pulses on path 68a cause this pointer register to cycle throughthe internal memory of buffer 52a and load successive bytes presented onpath 21a into the buffer locations specified by the internal pointerregister.

When path 65 carries a logical 0, buffer 52a is set to read mode andplaces on data path 63a a signal encoding the contents of the bytelocation addressed by the pointer register. As the pointer registercontent is incremented by pulses on path 68a, path 63a successivelycarries signals encoding each byte stored in buffer 52a. Further, whenbuffer 52a first enters read mode from write mode, the correction partof the ECC algorithm by which the ECC information appended to the dataon path 21a is developed, is implemented within buffer 52a to correctthe data in buffer 52a if necessary and possible. Similar activity isassociated with each of sub-block buffers 52b, c, d.

ECC test element 57a is very closely related to sub-block buffer 52a,and receives the data and byte parity signals on path 21a to perform thecomplementary function of detecting errors in the data. Errorsdetectable but uncorrectable by the ECC algorithm are independentlysignalled by ECC test element 57a with a logical 1 on path 67a. Alogical 0 indicates either a sub-block which had no errors in it or onein which errors had been corrected within buffer 52a. Test elements 57b,c, d are similarly related to buffers 52b, c, d and perform the samefunctions, providing a logical 1 signal on paths 67b, c, d whendetectable but uncorrectable errors are present in the sub-block justreceived, and a logical 0 otherwise. It is necessary to reset each testelement 57a, b, c, d before receipt of each sub-block.

A read operation requested by a signal on path 25 prompts control logicdevice 50 to execute a signal sequence for first loading the individualsub-blocks from DSUs 19a, b, c, d into buffers 52a, b, c, d and theneventually placing the sub-block bytes sequentially on paths 62a, b, c,corrected as necessary and possible. Initially, control logic device 50places a reset signal on path 66 which sets the internal pointerregisters in sub-block buffers 52a, b, c, d to the address of the firstbyte's location in each. It can be assumed that shortly thereafter DSUs19a, b, c, d (FIG. 1) start transmitting bits serially on paths 16a, b,c, d which are assembled into bytes and encoded in the signals on paths21a, b, c, d, each byte being followed shortly by a load clock (LD CLK)signal on paths 22a, b, c, d, all respectively.

Each LD CLK signal on the paths 22a, b, c, d is applied to one input ofan OR gate 55a, b, c, d respectively which in response produces theclock pulses on paths 68a, b, c, d needed to increment the pointerregisters in buffers 52a, b, c, d. Since the timing of the LD CLKsignals is ultimately controlled by the DSUs 19a, b, c, d individually,each buffer 52a, b, c, d can be filled at the speed of its associatedDSU 19a, b, c, d.

As each data or row parity sub-block byte on data paths 21a, b, c, d isreceived by buffers 52a, b, c, d, the byte is also transmitted to therespective ECC test element 57a, b, c, d. Before the start oftransmission from DSUs 19a, b, c, d, ECC test elements 57a, b, c, dreceive on path 54 the clear error data signal from control logic device50 which signal is used to initialize each element. Each test element57a, b, c, d has an internal accumulator which contains duringtransmission of data bytes to it, the current results of the errordetection algorithm employed by the elements 57a, b, c, d, and this isinitially set to 0 in each by the clear error data signal on path 54.Elements 57a, b, c, d also typically contain an internal counter, eachof which is set to the number of bytes in a data sub-block by the signalon path 54.

Each signal pulse on path 22a, b, c, d causes its associated ECC testelement's counter to decrement by 1. When the counter has beendecremented the number of times equalling the number of bytes in asub-block, the error test element 57a, b, c, d then uses the remainingbytes received as the error detection code and compares it to thecontents of the associated internal accumulator to determine whetherdetectable but not correctable errors are present in the datatransmitted on the associated path 21a, b, c, d. If no such errors arepresent in this data (or in the row parity information on path 21d) alogical 0 is placed on the associated output path 67a, b, c, d. If anerror is detected in this procedure, a logical 1 is placed on the path67a, b, c, d associated with the erroneous data or row parity.

As previously mentioned, there are several errors which can be sensedinternally by the DSUs 19a, b, c, d, and whose occurrence is signalledon the associated fault signal line 23a, b, c, d. The ECC test errorssignalled on paths 67a, b, c, d are provided with the DSU fault signalson paths 23a, b, c, d to the inputs of OR gates 82a, b, c, drespectively. OR gates 82a, b, c, d thus provide an output signal whichis a logical 1 when an error has been detected by either the associatedDSU 19a, b, c, d or the associated ECC test element 57a, b, c, d. The ORgate 82a, b, c, d, outputs form the inputs to the set (S) inputs offlip-flops 59a, b, c, d respectively on paths 85a, b, c, d.

At the start of each read operation flip-flops 59a, b, c, d receive ontheir reset (R) inputs the clear error data signal provided on path 54.This signal sets the initial state of the flip-flops 59a, b, c, d totheir cleared condition, where the logic levels of their outputs are 0.In response to a logical 1 on any of paths 85a, b, c, d, the associatedflip-flop 59a, b, c, d output on path 70a, b, c, d is set to alogical 1. Thus, after each group of sub-blocks comprising a data blockhave been loaded into buffers 52a, b, c, d, the outputs of flip-flops59a, b, c, d indicate by a 0 or a 1 at their outputs whether the datasub-block in the associated buffer 52a, b, c, d is respectively corrector in error. It should be noted that the logic circuitry handling therow parity sub-block stored in the row parity buffer 52d has somesimilarity to the logic circuitry for handling the data sub-blocks.

When the data sub-blocks and the row parity sub-block have been loadedinto the sub-block buffers 52a, b, c, and 52d respectively, and theerror flip-flops 59a, b, c, d have been set to indicate whether asub-block contains an error or not as just explained, then the remainderof the read process, including error correction if necessary, canproceed. The control logic device 50 resets the pointers in sub-blockbuffers 52a, b, c, d to the start of the sub-blocks again within thesebuffers. Control logic device 50 also sets the outputs on the R/W SELpath 65 to a logical 0, conditioning buffers 52a, b, c, d to output thedata stored in them on paths 63a, b, c, d. Control logic device 50 thenissues read clock (RD CLK) pulses at a preset rate on path 64 in anumber equal to the number of bytes stored in a sub-block. These arereceived by a second input terminal of OR gates 55a, b, c, d. Each ofthese pulses cause the OR gates 55a, b, c, d to transmit a pulse onpaths 68a, b, c, d respectively, causing buffers 52a, b, c, d totransmit one sub-block byte stored within each of them on paths 63a, b,c, d. Each set of data bytes from buffers 52a, b, c, and the row paritybyte from buffer 52d which issue in response to the same read clockpulse on path 64 contains associated information for purposes ofcorrecting a portion of the data according to this invention. It shouldbe noted that buffers 52a, b, c, d may be of the type which can bewritten into and read from simultaneously, in which case the buffers52a, b, c, d may be loaded by the next block to be read from DSUs 19a,b, c, d while the current block is undergoing any correction needed andtransmission from the buffers.

Transverse parity generator 56 simultaneously receives the data andparity bytes which have been read from buffers 52a, b, c, d by the sameread clock pulse on path 64, and in response to this data generates,properly ordered, the eight bits of the bit by bit odd parity of eachset of four associated bits provided on paths 63a, b, c, d. That is, thebits from each of the bytes on paths 63a, b, c, d which occupy the sameposition in their respective bytes are used to generate the bit in theparity byte on path 81 occupying the corresponding location. Odd parityis generated in each position so that if the bits involved are allcorrect, then the corresponding output parity bit on path 81 is alogical 0. If the parity of the four input bits is even, i.e., has oneincorrect bit in it, then generating odd parity provides a logical 1 onpath 81 in the corresponding bit position.

8×2 bit AND gate array 78 receives the 8 bits carried in parallel onpath 81, properly ordered, at its 8 data (D) inputs and the output ofinverter (I) element 74 on path 88 at each of its 8 gate (G) inputs. Ifthe signal on path 88 at the gate input is a logical 0, each bit of the8 outputs on path 69 from AND gate 78 is also a logical 0. If the signalon path 88 is a logical 1, the 8 data bits provided on path 81 to the 8data inputs of AND gate array 78 are gated to the outputs on path 69making its signal identical to the signal on path 81. It will beexplained later how the gate input on path 88 is set to a logical 1 ifthe parity information byte currently being processed appears to becorrect.

Turning next to the byte parity test elements 76a, b, c, d, each ofthese sequentially receive the bytes placed on paths 63a, b, c, d by therespective sub-block buffers 52a, b, c, d. The parity of each such byteis tested by the byte parity test element 76a, b, c, d receiving it, andif correct, a logical 0 is provided on the associated path 87a, b, c, dto the OR gate 77a b, c, d receiving the path's signal as an input. Ifparity is tested to be incorrect, then a logical 1 is provided on path87a, b, c, d respectively to the OR gate 77a, b, c, d involved. Asdescribed above, each OR gate 77a, b, c, d receives as its other inputthe output of the associated error flip-flop 59a, b, c, d.

The outputs of OR gates 77a, b, c are provided on paths 80a, b, crespectively to the 8 gate (G) inputs of each of the 8×2 bits AND gatearrays 60a, b, c. 8×2 bit AND gate arrays 60a, b, c are identical inconstruction to that of 8×2 bit AND gate array 78 and of course operatein the same way. 8×2 bit AND gate arrays 60a, b, c receive at their 8(D) inputs the properly ordered 8 bit output of 8×2 bit AND gate array78 on path 69. The 8 bit outputs of the AND gate arrays 60a, b, c onpaths 71a, b, c respectively thus duplicate the 8 bits on path 69 if andonly if the sub-block associated with the 8×2 bit AND gate array 60a, b,c involved has an error in it as indicated by a logical 1 carried on therespective input path 80a, b, c.

OR gate 77d receives the output of flip-flop 59d on path 70d and ofparity test element 76d on path 87d at its two inputs. If either or bothof these inputs is a logical 1, i.e. an error has been sensed asindicated by flip-flop 59d or detected by byte parity test element 76d,then OR gate 77d produces a logical 1 encoded in the signal at itsoutput, path 80d. The output of OR gate 77d is inverted by inverter 74and provided to the gate input of 8×2 bit AND gate array 78 on path 88.Thus, if the parity information byte on path 81 has passed all of itserror tests, a logical 1 is placed on path 88 and the parity informationbyte is gated by 8×2 bit AND gate array 78 to path 69.

8×2 bit exclusive OR (XOR) gate arrays 61a, b, c each receive twoproperly ordered 8 bit parallel inputs on their two inputs and providethe bit by bit exclusive OR of these two inputs as their outputs. As iswell known, an exclusive OR element generates a logical 0 value if thetwo input arguments or signals are equal to each other, and a logical 1value if the two arguments are unequal. Thus for each bit which is abinary or logical 1 in any of the 8 bit parallel paths 71a, b, c, 8×2bit XOR gate arrays 61a, b, c provide the inversion of the correspondingbit of the data sub-block bytes carried on paths 63a, b, c as the outputin the corresponding bit positions of 8 bit parallel data paths 62a, b,c. All of the bit values on paths 63a, b, c for which the correspondingbit values on paths 71a, b, c are a logical or binary 0, are providedunchanged in the corresponding bit position of the data paths 62a, b, c.To reiterate, it is, of course, essential that proper order of bitpositions in path 63a with path 71a, path 63b with path 71b, etc. bemaintained.

Thus, if a row parity error is present in a set of bits occupying thesame relative position in buffers 52a, b, c, d and one of the drives(via fault signals on paths 23a, b, c, d), byte parity tests (via paritytest elements 76a, b, c), or ECC test (elements 57a, b, c) identifiesthe buffer in which the erroneous bit is located, the bit is inverted bythe 8×2 bit XOR gate 61a, b, c receiving it on the respective path 63a,b, c. This corrects that bit in that its changed value causes itsassociated bits in the remaining two of the three buffers 52a, b, c androw parity buffer 52d to agree paritywise.

An example is helpful here. Assume that during readback of a data blockfrom DSUs 19a, b, c, d an error is detected in sub-block 2 by sub-block2 ECC test element 57b. This causes error flip-flop 2 (FF2) 59b to setset, with a logical 1 present on its output path 70b. At some time whileindividual 8 bit bytes are issuing on paths 63a, b, c, d further assumethat transverse parity generator 56 provides an output on path 81 inwhich a single bit is set to a logical 1. Let us assume that the databit corresponding to this logical 1 on path 81 and carried on path 63bis also a logical 1. If a logical 0 is present on path 80d indicatingthat according to conditions controlling its value the row paritysub-block in the row parity buffer 52d is correct, then the parity byteon path 81, including at least one logical 1 bit generated by the paritygenerator 56 and identifying the location of the bit in error on path63b, is gated to path 69. This 8 bit byte is further gated by thelogical 1 generated on path 80b by OR gate 77b to path 71b. The bit onpath 63b having the same bit position as the logical 1 on path 71b from8×2 bit AND gate 60b is inverted by the 8×2 bit XOR gate 61b and issuesas a logical 0 on path 62b because both inputs at the bit position havethe same value, in this case 1. The logical 0 on path 62b at theposition of interest here is the inverse of the logical 1 on path 63bwhich was read from DSU 19b. In all likelihood, this bit (and perhapsothers as well in this sub-block stored in buffer 52b) is incorrect, andby inverting this bit from buffer 52b, the correct value for the bit isencoded in the signal on path 62b. Note that inverting a single bit inany group of four for which parity is calculated by transverse paritygenerator 56 changes the parity of that group, on effect correcting it.

The unlikely event of two or more data and row parity sub-blocks of ablock being in error is dealt with by supplying the outputs from ORgates 77a, b, c, d to "2+bad sub-blocks" element 72. If two or morelogical 1's are presented on path 80a, b, c, d to element 72, thisindicates that two or more of the sub-blocks of a block have errors inthem. In response to this condition, element 72 provides a signal onpath 73 which indicates to the CPU or other external device thatuncorrectable errors are present in the block.

Note that for byte parity errors detected by parity test elements 76a,b, c, d, it is possible that for successive bytes, different datasub-blocks may contain the error(s), and yet be correctable. This isbecause the byte parity generated by the byte parity generators 18a, b,c, d shown in FIG. 1 is localized to the single byte involved, and henceneed not affect the correction of similar errors occurring innon-associated bytes in other sub-blocks. Note also that if a DSU or ECCfault is detected for a particular sub-block as indicated by theappropriate error flip-flop 59a, b, c, d, a byte parity error in adifferent sub-block can no longer be corrected. This condition isflagged by test element 72.

It is well known that the function of logic circuitry such as thatdescribed above can be duplicated by many different logic structures.For example, selection of logical 0 and 1 values is somewhat arbitraryin that these terms really represent only voltage levels and individualcircuit responses to these voltages. These conventions and others aswell are well known to those having familiarity with logic design, andno particular note need be taken of such.

As was previously mentioned, it is also important to realize that use ofthree only DSUs 19a, b, c to store data is probably not the number thata typical commercial system would have, since the reliability of theseunits justifies in most cases that eight or more be united in a singlesystem. The configuration of sub-block buffers 57a, b, c, d in storing 8bit parallel bytes is arbitrary as well. In such a complex electronicsystem as is described above, it is to be expected that manyalternatives are possible in employing the inventive concepts to providea device having similar capabilities. Thus, I respectively request thatthe claims here following be given an interpretation which covers mereimitations of the system described above and differ therefrom ininsubstantial ways while using my inventive concepts.

What I claim is:
 1. A data storage system for storing a data blocksupplied to the system encoded in a set of at least three individualdata sub-block signals, each of said data sub-block signals encoding oneof a number of sub-blocks into which the data block is broken, thenumber of sub-blocks equaling the number of data sub-block signals, andthe system comprising:(a) a plurality of similar data storage units innumber equal to the number of data sub-block signals and another,redundant, data storage unit, each capable of storing a plurality ofsub-blocks within itself responsive to a write select signal andencoding them in an output data sub-block signal responsive to a readselect signal specifying the sub-block desired, each of the datasub-block signals being received by a preselected one of the datastorage units, and each data storage unit being substantially physicallyindependent from the others and liable to independent failure to supplyoutput data sub-block signals correctly encoding the sub-blocks storedtherein; (b) data redundancy means receiving the set of data sub-blocksignals for generating according to a preselected algorithm a redundantdata sub-block, said redundant data sub-block being of the type forwhich a data reconstruction algorithm exists permitting reconstructionof any one data sub-block using the remaining data sub-blocks and theredundant data sub-block, and for providing to the redundant datastorage unit a signal encoding the redundant data sub-block; (c) aplurality of data storage unit error detection means, each operativelyassociated with a preselected data storage unit, for providing asub-block error signal responsive to failure of the associated datastorage unit to correctly encode in an output data sub-block signal asub-block stored within it; (d) control means for supplying the writeand read select signals to the data and redundant data storage units ina preselected sequence; and (e) data reconstruction means receiving thesub-block error signals from the error detection means and the data andredundant data sub-block signals from the data storage units, foremploying the data reconstruction algorithm to recreate and encode inthe output data signal, the data sub-block originally stored within thedata storage unit for which the sub-block error signal was generated. 2.The system of claim 1, further comprising:(a) byte error code generatingmeans receiving a data sub-block signal from a data storage unit, forproviding to the error detection means a signal encoding a byte errordetection code associated with at .[.last.]. .Iadd.least .Iaddend.onebyte within the sub-block, said byte error detection code generatedaccording to a preselected byte error detection algorithm which includesas a part thereof steps by which certain data errors in each said bytemay be detected; and (b) .Iadd.wherein the error detection meansincludes .Iaddend.byte error detection means receiving each byte errordetection code signal and a signal encoding the byte for which it wasgenerated, for applying the error detection algorithm to each said byteand its associated byte detection code and .Iadd.in .Iaddend.response todetection of an error in said byte, issuing a byte error signalassociated with the byte having the error and its sub-block; wherein thedata redundancy means further includes means for generating according tothe preselected algorithm a redundant data sub-block whose datareconstruction algorithm permits reconstruction of a byte within a datasub-block using an associated byte in the redundant sub-block and anassociated byte from each of the other data sub-blocks; and wherein thedata reconstruction means further includes means receiving the byteerror signal, for employing the data reconstruction algorithm torecreate and encode in the output data signal the data sub-block byteoriginally stored within the data storage unit.
 3. The system of claim2, wherein the byte error detection means receives signals encodingbytes from at least two different sub-blocks of the same data block andsupplies byte error detection code signals for each, and wherein thedata reconstruction means includes means receiving the byte errordetection code signals, for supplying an uncorrectable error signalresponsive to detecting errors in at least two bytes occupying the samerelative position in two different sub-blocks.
 4. The system of claim 2,wherein the byte error code generating means includes means forgenerating a parity bit signal for the byte.
 5. The system of claim 1,wherein the data reconstruction means further includes means forreceiving the sub-block error signals, and responding to errors in atleast two sub-blocks of the same data block, supplies an uncorrectableerror signal.
 6. The .[.apparatus.]. .Iadd.system .Iaddend.of claim 1wherein the data redundancy means includes means for generating for aset of associated bits, one from each data sub-block, a signal encodingthe parity of said set of bits. .Iadd.
 7. A fault tolerant data storagesystem comprising:a plurality of data storage units; means fororganizing data applied to the data storage system for storage into setsof data sub-blocks; means for generating a redundant data sub-blockassociated with the data sub-block organized by the organizing means;means for supplementing each data and redundant data sub-block witherror correction code syndromes; means for storing each supplementeddata sub-block and redundant data sub-block to a different one of thedata storage units as a data record; means for reading the data recordsand generating recovered, supplemented data sub-blocks and redundantdata sub-blocks; first stage data recovery means operating on the errorcorrection code syndromes and their associated recovered data andredundant data sub-blocks for correcting random errors in the respectiverecovered sub-blocks; sub-block error detection means operating on theerror correction code syndromes and their associated recovered data andredundant data sub-blocks for generating sub-block error signalsindicating recovered sub-blocks having errors exceeding the capacity oftheir associated error correction codes; and second stage data recoverymeans operating on a first sub-block error signals and the on therecovered data and redundant data sub-blocks for which no sub-blockerror signals are present for regenerating the data or redundant datasub-block for which the first sub-block error signal was generated..Iaddend. .Iadd.8. The fault tolerant data storage system as set forthin claim 7, wherein data is applied to the storage system in data blocksof predetermined length and wherein the organizing means divides theblock into data units and assigns an equal number of units to each ofthe plurality of data sub-blocks. .Iaddend. .Iadd.9. The fault tolerantdata storage system of claim 8, wherein the sub-block error detectionmeans further comprises processing error detection means for detectingerrors introduced to the sub-blocks after reading of records from thedata storage units, the processing error detection means including: aparity generator receiving recovered sub-blocks and generating a paritybit for each data unit and adding the parity bit to the data unit; and aparity test unit receiving data sub-blocks from the first stage datarecovery means, checking parity of each data unit of the respectivesub-blocks and causing generation of a sub-block error signal associatedwith each defective data unit. .Iaddend. .Iadd.10. The fault tolerantdata storage system as set forth in claim 8, wherein the redundant datasub-block generating means generates a redundant data sub-blockcomprising data units and having the same number of units as the memberdata sub-blocks of the set. .Iaddend. .Iadd.11. The fault tolerant datastorage system as set forth in claim 10, wherein the data storage unitshave corresponding address ranges and wherein the storage means storescorresponding data units of a set of data sub-blocks and the associatedredundant data sub-block to the same addresses in different data storageunits. .Iaddend. .Iadd.12. The fault tolerant data storage system as setforth in claim 11, wherein the data storage units are synchronized,fault independent disk drive units. .Iaddend. .Iadd.13. The faulttolerant data storage system as set forth in claim 12, wherein eachfault independent disk drive unit further includes means for generatinga disk fault signal and wherein the first stage data recovery meansoperates on a disk fault signal to generate a sub-block error signal forthe sub-block stored as a record on the affected disk drive unit..Iaddend. .Iadd.14. The fault tolerant data storage system as set forthin claim 7, wherein the data organizing means operates on a data blockof predetermined size applied to the system for storage to form datasub-blocks by transferring operative data units from the data block insequence to each of the sub-block error correction code syndromesupplementing means in turn. .Iaddend. .Iadd.15. The fault tolerant datastorage system as set forth in claim 14, wherein the data record readingmeans further includes deserializer means for reassembling the dataunits of the recovered data and redundant data sub-blocks. .Iaddend..Iadd.16. The fault tolerant data storage system as set forth in claim15, wherein the sub-block error detection means further includes errordetection code test means receiving the recovered, supplementedsub-blocks for determining whether error occurring in each of therecovered sub-blocks is correctable by the random error correcting meansand causing generation of a sub-block error signal when a sub-block hasuncorrectable error. .Iaddend. .Iadd.17. The fault tolerant data storagesystem as set forth in claim 16, wherein the second stage data recoverymeans further includes means responsive to two or more sub-block errorsignals for signalling a condition of uncorrectable error. .Iaddend..Iadd.18. A data recovery system operating in a data write/read channelto and from a plurality of synchronized, fault independent disk driveunits, the data recovery system comprising:a data block dividerorganizing a data block received over a data transmission channel into agroup of data sub-blocks; a parity generator operating on the group ofdata sub-blocks and generating a redundant data sub-block associatedwith a group of data sub-blocks; a plurality of error correction codegenerators, each operating on one of the data and redundant datasub-blocks of a group to supplement each data and redundant datasub-block with error correction code syndromes; each disk drive unitbeing coupled to one error correction code generator to receive asupplemented data or redundant data sub-block for storage as a datarecord; means for reading the data records and generating recovered,supplemented sub-blocks; a plurality of first stage data recoverybuffers receiving the recovered, supplemented sub-blocks and operatingon the error correction code syndromes and their associated sub-blocksfor correcting random errors in each recovered sub-block; a sub-blockerror detection unit including a plurality of error correction code testunits operating on each recovered sub-block to generate sub-block errorsignals indicating recovered sub-blocks having errors exceeding thecapacity of the error correction code syndromes associated therewith;and a second stage data recovery unit operating on a first sub-blockerror signal and the on the recovered data and redundant data sub-blocksfor which no sub-block error signals are present for regenerating thedata sub-block associated with the first sub-block error signal..Iaddend. .Iadd.19. The data recovery system as set forth in claim 18,wherein the reading means further comprises a data byte parity bitgenerator operating on groups of adjacent bits as a unit and appendingto the unit a parity bit. .Iaddend. .Iadd.20. The data recovery systemas set forth in claim 19 wherein the disk drive units generate faultsignals associated with independent failures of the disk drive units..Iaddend. .Iadd.21. The data recovery system as set forth in claim 20,wherein the sub-block error detection unit further comprises: means forgenerating a sub-block error signal in response to each disk drive unitfault signal; and a parity test unit associated with each first stagedata recovery buffer and operating on each byte with its associatedparity bit for generating a byte error signal upon detection of error..Iaddend. .Iadd.22. A fault tolerant data storage systemcomprising:means for receiving an input data stream and dividing thedata stream into a plurality of data columns; means for generating aparity data column; a plurality of data storage units; means fortransferring said data columns, including said parity column, into andout of differing ones of the data storage units, the means fortransferring including an error correction and detection interfaceassociated with each disk drive for correcting random errors within itserror correction capacity and signalling errors exceeding its correctioncapacity; and parity reconstruction means, responsive to an error signalfrom an error correction and detection interface, for reconstructingdata in a data column associated with the error correction and detectioninterface signalling the error. .Iaddend. .Iadd.23. A fault tolerantdata storage system such as that set forth in claim 22 wherein eacherror correction and detection interface further includes disk drivefailure detection and error signalling means. .Iaddend. .Iadd.24. Afault tolerant data storage system such as that set forth in claim 23wherein said means for transferring data columns into differing ones ofsaid disk drives further includes means for generating an errorcorrection code for each data column and transferring the errorcorrection code along with the data column into the disk drive..Iaddend. .Iadd.25. A disk data storage system for storing data blocksapplied to the system in the form of electrical signals, the disk datastorage system comprising: means for dividing data blocks into datasub-blocks; means for generating a parity data sub-block from the datasub-blocks; means for generating and appending an error correction codeto the each data sub-block, including the parity data sub-block; aplurality of disk data storage subsystems for storing data; means forstoring each data sub-block to corresponding locations in the individualdisk data storage subsystems in parallel; means for reading datasub-blocks from the disk data storage means and for producing recovereddata sub-block signals in parallel; and data recovery means to which therecovered data sub-block signals are applied, the data recovery meansbeing operable on the signals corresponding to the data digits, paritydigits and error correction codes to produce electrical signalscorresponding to the data digits of data blocks applied to the disk datastorage system for storage. .Iaddend. .Iadd.26. A disk data storagesystem as set forth in claim 25 wherein the means for generating aparity data sub-block includes means coupled to the means for dividingdata blocks for receiving the data sub-blocks, row by row, forgenerating row parity bits and assembling the row parity bits into aparity sub-block. .Iaddend. .Iadd.27. The disk data storage system asset forth in claim 26, wherein individual data units from each sub-blockare stored in correlated locations of their respective disk subsystemsas data rows, each data row having one bit from each data sub-block andat least one parity bit from the parity data sub-block to aid inparallel recovery. .Iaddend. .Iadd.28. A disk data storage system as setforth in claim 26 and further comprising means for indicating a disksubsystem fault, the data recovery means being responsive to indicationof a disk subsystem fault to utilize the data sub-block signals andparity sub-block signals to reconstruct the data from the disk subsystemindicating fault and responsive to two or more simultaneous indicationsof disk subsystem fault to cause generation of a signal indicating datanot recoverable. .Iaddend. .Iadd.29. The disk data storage system as setforth in claim 27 wherein the disk data storage subsystems comprise atleast a first disk each, the disks being synchronized to facilitateparallel recovery of correlated data bits. .Iaddend. .Iadd.30. A faulttolerant disk data storage system for storing data blocks applied to thesystem for storage, each data block having a plurality of data segments,comprising:means for organizing an applied data block into a pluralityof data sub-blocks, the data sub-blocks being exclusive sets of datasegments; means for defining data rows of exclusive sets of datasegments, each data row including a first data segment from each datasub-block; means for generating at least a first redundant data segmentassociated with each data row, the redundant data segments being fewerin number than the number of data segments in the data block; means fororganizing at least a first redundant data sub-block, each redundantdata sub-block including at least a first segment of redundant dataassociated with each data row; means for generating error correctioncodes for each data sub-block and redundant data sub-block and appendingsaid error correction codes to their respective data sub-blocks; a diskdata storage subsystem for each data sub-block and redundant datasub-block; means for storing each data sub-block and each redundant datasub-block for an applied data block to a separate disk data storagesubsystem; means for reading data sub-blocks and redundant datasub-blocks associated with a stored data block from the disk subsystemsand generating logic signals corresponding to a retrieved data block,the redundant data sub-block and the error correction codes; and meansutilizing all available logic signals associated with the retrieved datablock for generating a restored data block, the data segments of therestored data block and the data segments of a data block applied to thesystem for storage being the same notwithstanding partial failure inretrieval and for indicating a data unrecoverable condition. .Iaddend..Iadd.31. A fault tolerant disk data storage system for storing appliedgroups of data segments, the system comprising:means for organizing eachapplied group of data segments into a array of data rows and datacolumns, with each data segment belonging to one row and to one columnof the array; means for generating at least a first redundant datasegment associated with each data row, the redundant data segments beingfewer in number than the number of data segments in the group; means fororganizing at least a first redundant data column, each redundant datacolumn including at least a first segment of redundant data associatedwith each data row; means for generating error check digits for eachdata column and redundant data column and appending said error checkdigits to their respective columns; a disk data storage subsystem foreach data column and redundant data column; means for storing each datacolumn and redundant data column for a group of data segments to aseparate disk data storage subsystem; means for reading the datasegments and redundant data associated with a group of data segments andstored in the disk data storage subsystems and generating logic signalscorresponding to a retrieved group; and means for executing an algorithmutilizing all available logic signals associated with the retrievedgroup for generating a restored group of data segments, the datasegments of the restored group and the data segments of the appliedgroup of data segments being the same notwithstanding certain partialfailures in retrieval.